'.\" t
.TH "clstat" "1M" "Jun 27, 2006" "1\&.2\&.0"
.SH NAME
clstat \- Linuxha.net Cluster Status

.SH SYNOPSIS
.TS
l.
clstat [\fB-V|--verbose\fP] [\fB--nochecksums\fP]
       Show overview of cluster status 

clstat \fB-L|--locks\fP [\fB-V|--verbose\fP] [\fB--nochecksums\fP]
       Show Locking information for cluster nodes

clstat \fB-N|--nets\fP [\fB-V|--verbose\fP] [\fB--nochecksums\fP]
       Show Networking information for cluster nodes

clstat \fB-M|--modstats\fP \fB-A|--application\fP \fIapp\fP [\fB-V|--verbose\fP] [\fB--nochecksums\fP]
       Show Lems Module stats for Application 

clstat \fB-A|--application\fP \fIapp\fP [\fB-V|--verbose\fP] [\fB--nochecksums\fP]
       Show detailed information for a Linuxha.net application

clstat \fB-?\fP
       Show brief usage information
.TE

.SH DESCRIPTION
This utility is used to view the status of the cluster as a whole, the
status of various subsystems or to give detailed information about 
a particular application. 

This command can only be run once the cluster has been successfully 
built using the \fIclbuild(1M)\fP utility. Please note that if you change 
the cluster configuration file without running \fIclbuild(1M)\fP again 
this utility will error without use of the \fB--nochecksums\fP option.

Simialrly if the \fB--application\fP option is used and the configuration
of the application as defined by the \fIappconf.xml(5)\fP has changed then
if the \fIclbuildapp(1M)\fP tool has not been run for the application, again
the \fB--nochecksums\fP will be needed.

.SH SUMMARY INFORMATION
When the software is run in summary mode the information shown simply
indicates whether the cluster is running, and if it is running, the status
of each node in the cluster. Typically you might see something similar to
the following:

.TS
l.
clstat output goes HERE
.TE

The information available when the cluster is not running should still 
indicate the name, nodes and application details, and hence a typical
output when no applications are defined would be:

.TS
l.
clstat when cluster not running HERE
.TE

If a cluster host has been part of the cluster, but the cluster daemon appears
not be to be functioning then the output may look similar to the following:

.TS
l.
clstat when cldaemon died HERE
.TE

In this situation the node "nodeB" is still up and running, but the 
cluster daemon on that node appears to have died unexpectedly. 

When any applications have been built then a per-application summary line
will appear in the following format:

.TS
l.
 Application       Node      State  Running  Monitor  Stale  Fail-over?
       samba    serverc    STARTED  0:00:01  Running      3         Yes
.TE

Each field will now be explained:
.TP 4
.B Application
This is the name of the application. It should be limited to alphanumeric
characters, no spaces and preferrably in lower case. This is no length limit
on the same but 10 characters maximum is recommended.
.TP
.B Node
This defines the node where the application is currently running. If the
application is not running then this field will be populated with "N/A".
.TP
.B State
This indicates that status of the application, and it will be one of "STOPPED",
"STARTING", "STOPPING" or "RUNNING". An explanation of each term can be
found in the \fBApplication Status\fP sub-section below.
.TP
.B Running
If the application is running, (even if it is currently stopping), this
indicates the length of time the application has been running on this node.
The time is given in the format: days:hours:minutes.
.TP
.B Monitor
This will be "Running", "Stopped" or "N/A" - this indicates whether there is
currently a lems daemon running for the application in question.
.TP
.B Stale
This should be zero, if greater than zero it indicates the number of 
file systems that are currently not synchronised, and thus are affecting 
availability in a fail-over scenario.
.TP
.B Fail-over?
This indicates whether the application will fail-over to the other node in
the event of a software problem. This will be set to "no" if the other node
is not running, or the application has failed-over from this node due to a
software problem (and the valid node list for the application has not yet
been reset).
.RS

.SS Application States
The application states should be quite self explanetory, but just in case:

.TP 4
.B STOPPED
The application is not currently running, and is not in the process of 
starting to run. 
.TP
.B STARTING
The application is starting to run on the specified server. This state 
exists as long as it takes for all checks to be performed, all servers
and services to be started and finally for the application and application
monitors to start. 

Thus this status typically exists for less than a minute, before
changing to the "Running" state.
.TP
.B STOPPING
Indicates that the application is stopping on the specified node. This is
either due to the administrator stopping the application specifically, or
the  application monitor indicating that the current server is no longer
suitable to host this application. In this instance it will change from
"Stopping" to "Starting", but the "Starting" state will only exist until
the application is running successfully on the other node.
.TP
.B RUNNING
This is the normal state of an application - it indicates that the application
is operating on the specified server and is not in the process of stopping
or starting.
.RS

.SS Unregistered Applications
If an application has been previously built, but the current application
configuration has not been rebuilt such an application can not be started
in the cluster. 

These applications will still appear in the \fIclstat(1M)\fP output in the
following format:

.TS
l.
 Application    Node   State  Started  Monitor  Stale  Fail-over?
        test     N/A    DOWN      N/A      N/A    N/A         N/A  [Rebuild]
.TE

The "[Rebuild]" indicates that the \fIclbuildapp(1M)\fP process
needs to run against the application before it can be used - see the
\fIclbuildapp(1M)\fP manual page for further details.

.SH DETAILED LOCK INFORMATION
When the \fB--locks\fP option is used then locking information is shown.
This option can be combined with \fB--application\fP, \fB--modstats\fP 
or \fB--nets\fP to shown more than one type of information if desired.

Lock management is local to each node, and hence the lock details are
given include the name of the node the lock applies to.

.TS
l.
SAMPLE LOCK OUTPUT
 Node     Lock Name      Program        PID   Gained at    Release at
 server1  NET
 server1  NBD_CLIENT
 server1  NBD_SERVER
 server2  NET
 server2  NBD_CLIENT
 server2  NBD_SERVER
.TE

If the cluster is not currently running only lock information for the 
local node will be shown. The format of the locks section of output
in this case is as follows:

.TS
l.
     Node Lock Name      Program     PID   Gained at    Release at
slack10s1 NET            N/A         N/A   N/A          N/A
slack10s1 NBD_CLIENT     N/A         N/A   N/A          N/A
slack10s1 NBD_SERVER     N/A         N/A   N/A          N/A
.TE

For more information on the types of the locks available please see the
\fIcllockd(1M)\fP manual  page.

.SH DETAILED NETWORK INFORMATION
When the command line includes the \fB--nets\fP option networking information
as defined by a running network daemon on each node is shown. Assuming that 
both nodes are running the output may appear similar to the following:

.TS
l.
 Network status for server1.linuxha.net

    Network   Status  Link Check?   Interface  Monitor?
       drbd    alive          yes        eth2       yes
       main    alive          yes        eth0       yes

 Network status for server2.linuxha.net

    Network   Status  Link Check?   Interface  Monitor?
       drbd    alive          yes        eth2       yes
       main    alive          yes        eth0       yes
.TE

Note that network information relies on teh cluster network daemon running.
If it is not then no information can be generated.

Each field will now be explained:
.TP 4
.B Network
The name of the network as defined in the cluster configuration file.
.TP
.B Status
The network status - whether the daemon believes that any of the
cards available are 'live' and working. Will show the value 'alive'
or 'dead'.
.TP
.B Link Check?
Whether the interface currently in use support physical link level checking.
.TP
.B Interface
The interface that currently hosts the IP address for this network.
.TP
.B Monitor?
Will be 'yes' or 'no' - indicates whether the network daemon is monitoring the
network currently. (It is possible for an administrator to selectively disable
network monitoring when maintenance is required whilst the cluster is running).
.RE

.SH APPLICATION MODULE STATUS
If the \fB--modstats\fP argument is given then the modules running for the
\fBLems\fP daemon for this process are shown. This is only summary 
information - but describes which modules have been successfully loaded and
which have generated any errors in the process of running.

.TS
l.
clstat --modstats output HERE
.TE

.SH DETAILED APPLICATION INFORMATION
When displaying information for a particular application the output
format will firstly include a section similar to the following, 
(assuming the application is in the "Running" state). 

.TS
l.
 File Systems
 
 Mount Point               Valid   Type      State   % Complete   Completion
 /samba/cfg                 both   drbd       Sync
 /samba/logs               local   drbd     Unsync
 /samba/shares             local   drbd    syncing         18 %    1:00
.TE

If a 'lems' session is running for the application, (and it should be otherwise
a warning would have been shown when the application was started), then 
it is likely that additional output will be shown. The additional lines 
shown depend on the monitors you have configured, but are likely to look
like the following:

.TS
l.
  Process Monitors
 
  Name    Status  Restarts   Current          Reset at
  smbd   Running         1         0               N/A
 
  General Monitors
 
  Type                  Name   Status
  Flag Check      flag_check   Running
  FS Monitor       fsmonitor   Running
.TE

The above sample is merely that - an example of the monitors that might be
shown. Certain of the monitors might not be running, or might not currently
be in use - for example the "flag check" monitor.

The monitor typically come as either a "Process Monitor" or a "General Monitor".
One or more process monitors might be running, and these are responsible
for ensuring that the application in question is still running, and 
re-starting it, or failing the application over to the other node if
deemed necessary.

The General Monitors are typically used to keep track of the cluster
state according to this application, and make any changes as required. At
present the expected list of "General Monitors" might include the following.

.TP 8
.B Flag Check
This is a utility that checks for the existence of files in a certain 
directory and if present will pause the monitoring of the named monitor.
This is particularly useful if the user wishes to allow non-root users to
pause/resume monitors via script.

The directory where the files should be created is:

.TS
l.
/etc/cluster/<package>/flags
.TE

The name of the flag is simply the name associated with the monitor, 
for example "fsmonitor". Once the file has been touched it may take a
certain period before the action is taken, but after it has taken effect
the line for the monitor would be changed to:

.TS
l.
  FS Monitor       fsmonitor   Stopped
.TE

Removal of the file will resume the monitor after a short delay.

.TP
.B FS Monitor
This is the file system monitor - probably one of the most important
monitors. It is responsible for ensuring the status of the file systems
is communicated to the cluster daemons, and when possible for automatically
attempting re-synchronisation of this data if a node rejoins the cluster
after a failure.

.SH ARGUMENTS
Only a small number of options are currently avaiable currently.

.TP 8
.B -V,--verbose
Gives some output to the standard output regarding the operation of the
command - nothing of any real detail is currently produced, however.
.TP
.B -A,--application
Show verbose information about the specified application - if the application
is currently running in the cluster.
.TP
.B -L,--locks
Shows details of the locks for each node that is currently part of the
cluster.
.TP
.B -N,--nets
Include network status information in the output.
.TP
.B -M,--modstats
Show some details for the Lems modules that are loaded for the application
in question. When this option is used the \fB--application\fP must also be
specified.
.TP
.B --nochecksums
Normally if the cluster or application configuration files (if showing details
for an application), have been modified whilst the cluster is running the
checksums which are used to indicate the last sane and checked configuration 
will not be valid. In such instances many of the Linuxha.net commands, including
this will not will not function. If necessary the \fB--nochecksums\fP can be
used to overcome this until the cluster or application configuration are
next rebuilt.

.SH EXIT STATUS
The \fIclstat(1M)\fP utility makes use of many error codes, but in summary
it will return a non-zero number for an error or zero if the information
requested has been successfully processed.

.SH SEE ALSO
.TS
l l.
clbuild(1M)	- Build / Validate cluster topology
clbuildapp(1M)	- Build / Synchronise cluster application 
cllockd(1M)	- Cluster lock Daemon
clstat(1M)	- Show cluster status information
cldeamon(1M)	- Cluster status Daemon
clstartapp(1M)	- Start a clustered application
clrunapp(1M)	- High level cluster application execution
clhaltapp(1M)	- Halt a clustered application
clconf.xml(5)	- Overall cluster topology configuration file
appconf.xml(5)	- Configuration of an application used by the cluster
.TE

.SH NOTES
The detailed information presented for an application via the 
\fB--application\fP argument is mostly taken from the \fILems\fP 
deamon running for that application.
If such a process is not running then very little output will be produced.

When the cluster is performing the fail-over of an application, or has
just lost the other node in the cluster the response times for the
\fIclstat(1M)\fP can increase for a short period.

.SH AUTHOR
The \fIclstat(1M)\fP utility was written by Simon Edwards, 2003-2006. The
author can be contacted via the website mentioned below.

.SH AVAILABILITY
This software is freely available from the Linuxha website - please see
\fBhttp://www.linuxha.net\fP for more details.

.SH WARRANTY
This is Open Source Software is per the GNU GPL. It is free to use and
distribute but \fIcomes with no warranty whatsoever\fP. For more information
on the license please see \fBwww.gnu.org/copyleft/gpl.html\fP.

